Wherefore, what is claimed is: 

1 . A system for generating a high dynamic range (HDR) image from 
an image sequence of a scene captured while varying the exposure of each 

5 image, comprising: 

a general purpose computing device; 

a computer program comprising program modules executable by 
the computing device, wherein the computing device is directed by the program 
modules of the computer program to, 
1 0 designate the image having the greatest number of valid 

pixels as a reference image, wherein a pixel is valid if it is not saturated and it 
exhibits an acceptable degree of contrast, 

for each image except the reference image, register the 
image under consideration with the image, including the reference image, which 
15 exhibits an exposure that is both closer to that of the reference image than the 
image under consideration and closest among the images to the exposure of the 
image under consideration, to generate a flow field, 

concatenate the flow fields generated for the non-reference 
images not already registered with the reference image, to register each of them 
20 with the reference image, 

warp each non-reference image using the flow field 
associated therewith, and 

combine the reference image and the warped images to 
create a radiance map representing the HDR image. 

25 

2. The system of Claim 1 , further comprising a program module for 
tone mapping the radiance map to convert it into an 8-bit representation suitable 
for rendering. 

30 3. The system of Claim 2, wherein the program module for tone 

mapping the radiance map, comprises sub-modules for: 
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converting the radiance map to CIE space and recovering the 
chromaticity coordinates to produce a luminance image; 

compressing the dynamic range of the luminance image and re- 
inserting the chrominance; and 

5 converting the CIE space image to produce the final 8-bit RGB 

image. 

4. The system of Claim 1 , wherein the number of images in said 
sequence of images and the variation in the exposure among the images is such 

1 0 that the images collectively capture substantially all the brightness variation of 
the scene depicted in the images. 

5. The system of Claim 1 , wherein the intensity of a pixel is measured 
in terms of 8-bit RGB color space values, and wherein a pixel is considered as 

15 being unsaturated and exhibiting the acceptable degree of contrast if each of its 
RGB values is within a prescribed interval. 

6. The system of Claim 5, wherein the prescribed interval is between, 
and including, 17 and 254.. 

20 

7. The system of Claim 1 , wherein the sub-module for registering the 
image under consideration with the image exhibiting the exposure that is both 
closer to that of the reference image than the image under consideration and 
closest among the images to the exposure of the image under consideration, to 

25 generate a flow field, comprises sub-modules for: 

whenever the image under consideration has a shorter exposure 
than the other input image it is being registered with, 

boosting the intensity of the image under consideration to 
substantially match the intensity range of said other input image, 
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computing a flow field that globally registers the image under 
consideration with said other input image by estimating a global transform that 
maps the image under consideration onto that other image; and 

computing a dense motion field, which forms a local 
correction to the flow field computed with the global transform, using a gradient 
based optical flow; wherein 

the corrected flow field comprises a composite vector for 
each pixel location of the flow field which is the sum of (i) the global component 
derived from said global transform, and (ii) a local component taken from said 
dense motion field that forms the local correction for the global component. 

8. The system of Claim 7, wherein the sub-module for computing a 
dense motion field that forms a local correction to the global transform using a 
gradient based optical flow, comprises sub-modules for: 

employing a variant of the Lucas and Kanade technique used in a 
Laplacian pyramid framework wherein the image under consideration is warped 
towards said other input image and the residual flow vectors are estimated at 
each level of the pyramid; and 

accumulating the residual flow vectors computed for each pixel at 
each level of the pyramid to establish said local component of the dense motion 
field. 

9. The system of Claim 1 , wherein the sub-module for registering the 
image under consideration with the image exhibiting the exposure that is both 
closer to that of the reference image than the image under consideration and 
closest among the images to the exposure of the image under consideration, to 
generate a flow field, comprises sub-modules for: 

whenever the image under consideration has a longer exposure 
than the other input image it is being registered with, 

boosting the intensity of said other image to substantially 
match the intensity range of the image under consideration, 
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computing a flow field that globally registers the image under 
consideration with said other input image by estimating a global transform that 
maps the image under consideration onto that other image; and 

computing a dense motion field, which forms a local 
5 correction to the flow field computed with the global transform, using a gradient 
based optical flow; wherein 

the corrected flow field comprises a composite vector for 
each pixel location of the flow field which is the sum of (i) a global component 
derived from said global transform, and (ii) a local component taken from said 
10 dense motion field that forms the local correction for the global component. 

10. The system of Claim 9, wherein the sub-module for computing a 
dense motion field that forms a local correction to the global transform using a 
gradient based optical flow, comprises sub-modules for: 

1 5 employing a variant of the Lucas and Kanade technique used in a 

Laplacian pyramid framework wherein the image under consideration is warped 
towards said other input image and the residual flow vectors are estimated at 
each level of the pyramid; and 

accumulating the residual flow vectors computed for each pixel at 

2 0 each level of the pyramid to establish said local component of the dense motion 
field. 



1 1 . The system of Claim 1 , wherein the program module for combining 
the reference image and warped images to create the radiance map, comprises 
25 sub-modules for: 

converting the reference image and each warped image, 
respectively, to radiance images; and 

assigning a radiance value to each pixel location in the radiance 
map, wherein said radiance map radiance value is the radiance value assigned 
30 to the same pixel location in the radiance image associated with the reference 
image or a weighted combination of two or more radiance values taken from the 
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corresponding pixel locations in the radiance images associated with the 
reference image and the warped images, depending on which values are 
deemed to be trustworthy based on the intensity of the pixel at that pixel location 
in the reference image. 

1 2. A computer-implemented process for generating a high dynamic 
range (HDR) image from an image sequence of a scene captured while varying 
the exposure of each image, said process comprising using a computer to 
perform the following process actions: 
inputting said images; 

designating the input image having the greatest number of valid 
pixels as a reference image, wherein a pixel is considered valid if it is not 
saturated and it exhibits a prescribed degree of contrast; 

for each input image except the reference image, identifying 
corresponding pixels between the image under consideration and another input 
image, including the reference image, which exhibits an exposure that is both 
closer to that of the reference image than the input image under consideration 
and closest among the input images to the exposure of the input image under 
consideration; 

using the pixel correspondences identified between pairs of input 
images to establish a set of corresponding pixels among the input images for 
each pixel of the reference image; and 

for each set of corresponding pixels, identifying at least one pixel in 
the set that represents a trustworthy pixel and employing the pixel color 
information associated with the one or more identified trustworthy pixels to 
compute a radiance value for that set of pixels to form a radiance map 
representing the HDR image. 

13. The process of Claim 12, further comprising a process action of 
tone mapping the radiance map to convert it into an 8-bit representation suitable 
for rendering. 
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14. The process of Claim 12, wherein the process action of identifying 
corresponding pixels between the image under consideration and said other 
input image, comprises an action of computing a flow field that registers that 

5 image with said other input image. 

15. The process of Claim 14, wherein the process action of using the 
pixel correspondences identified between pairs of input images to establish a set 
of corresponding pixels among the input images for each pixel of the reference 

0 image, comprises an action of concatenating the flow fields generated for the 
non-reference images not already registered with the reference image to register 
each of them with the reference image. 

16. The process of Claim 14, wherein the process action of, for each 
3 set of corresponding pixels, identifying at least one pixel in the set that 

represents a trustworthy pixel and employing the pixel color information 
associated with the one or more identified trustworthy pixels to compute a 
radiance value for that set of pixels to form a radiance map representing the 
HDR image, comprises the actions of: 
) converting the reference image and each warped image, 

respectively, to radiance images; and 

for each of pixel location of the reference image, 

determining if the radiance values assigned to the 
corresponding location in the warped images are within a maximum allowable 
noise variance of the radiance value assigned to pixel location of the reference 
image under consideration, 

whenever it is found that at least one of the radiance values 
assigned to said corresponding location in a warped image is within the 
maximum allowable noise variance of the radiance value assigned to the pixel 
location in the reference image under consideration, computing a weighted 
average of radiance values assigned to the pixel location of the reference frame 
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under consideration and said corresponding pixel location in the warped images 
whose radiance values fell within the maximum allowable noise variance, and 
assigning the weighted average as the radiance value for the pixel location under 
consideration in the radiance map, and 

whenever it is found that none of the radiance values 
assigned to said corresponding locations in the warped images are with the 
maximum allowable noise variance of the radiance value assigned to the pixel 
location in the reference image, assigning the radiance value assigned to the 
pixel location under consideration in the reference image as the radiance value 
for the pixel location under consideration in the radiance map. 

1 7. The process of Claim 16, wherein the process action of computing 
a weighted average of radiance values assigned to the pixel location of the 
reference frame under consideration and said corresponding pixel location in the 
warped images whose radiance values fell within the maximum allowable noise 
variance, comprises an action of computing the weighted average radiance as 

fm* {Pr,Ps, )•• + -fwM {Pr, Ps,) + fm, (/>*> Pl, )■■ + J™ {Pr>Pl. ) + fw (Pr) 

wherein p^, P Si and p u are corresponding pixels in the radiance images with 
subscript R referring to the radiance image associated with the reference image, 
subscript S t referring to the radiance images associated with each of the warped 
images having an exposure shorter than the reference image and subscript L, 
referring to the radiance images associated with each of the warped images 
having an exposure longer than the reference image, and wherein the weighting 
function / m (q,p) = f M (\ p - q \)f w ( p ) j s an intensity-based weight function f w 
modulated by a plausibility map/ M , where M) is a Hermite Cubic defined by 
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^max J 



V < max anc j j S a prescribed parameter, 

0 otherwise 

such that the modulated weight of a radiance value associated with a warped 
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image that is outside said maximum allowable noise variance is set to zero when 
calculating the weighted average radiance. 



1 8. A computer-readable medium having computer-executable 
instructions for generating a high dynamic range (HDR) image from a sequence 
of bracketed images, said computer-executable instructions comprising: 

(a) inputting said bracketed images; 

(b) sorting the input images in the order of their exposures; 

(c) designating the input image having the greatest number of 
valid pixels as a reference image, wherein a pixel is valid if it is not saturated and 
it exhibits a prescribed degree of contrast; 

(d) selecting a previously unregistered non-reference input 

image; 

(e) determining whether the selected input image has an 
exposure that is shorter or longer than the reference image; 

(f) whenever the selected input image has an exposure that is 
shorter than the reference image, 

identifying the input image having the next shortest exposure 
that is still longer than the selected input image, which could be the registered 
image, 

boosting the selected image to match the exposure of the 

identified image, 

registering the boosted selected image with the identified 
image, to generate a flow field for the selected image; 

(g) whenever the selected input image has an exposure that is 
longer than the reference image, 

identifying the input image having the next longest exposure 
that is still shorter than the selected input image, which could be the registered 
image, 

boosting the identified image to match the exposure of the 

selected image, 
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registering the selected image with the boosted identified 
image, to generate a flow field for the selected image; 

(h) determining if there are any remaining unregistered non- 
reference input images, and if so repeating instructions (d) through (h) until all 
the non-reference images have been registered; 

(i) concatenating the flow fields generated for the non- 
reference input images not already registered with the reference image to 
register each of them with the reference image; 

(j) warping each non-reference input image using the flow field, 
or concatenated flow field if one, associated therewith; and 

(k) combining the reference image and the warped images to 
create a radiance map representing the HDR image. 

1 9. The computer-readable medium of Claim 18, further comprising an 
instruction for tone mapping the radiance map to convert it into an 8-bit 
representation suitable for rendering. 

20. The computer-readable medium of Claim 1 8, wherein the intensity 
of a pixel is measured in terms of 8-bit RGB color space values, and wherein a 
pixel is considered as being unsaturated and exhibiting the acceptable degree of 
contrast if each of its RGB values is between, and including, 17 and 254. 
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